Four-dimensional multi-plane broadband imaging system based on non-reentry quadratically distorted (nrqd) grating and grism

ABSTRACT

Disclosed is a four-dimensional (4D: 3D+time) multi-plane broadband imaging apparatus capable of recording 3D multi-plane and multi-colour images simultaneously. The apparatus comprises: one or more non-reentry quadratically distorted (NRQD) gratings ( 5 ) which can produce a focal length and a spatial position corresponding to each diffraction order, thus simultaneously transmitting wavefront information between multiple object/image planes ( 2 ) and a single image/object plane ( 7 ); a grism system ( 4 ) which can limit chromatically-induced lateral smearing by creating a collimated beam in which the spectral components are laterally displaced; a lens system ( 3 ) which is configured to adjust the optical path; and the optical detector(s) ( 6 ). In an optical system, the multiple object/image planes ( 2 ), the lens system ( 3 ), the grism system ( 4 ), the NRQD grating(s) ( 5 ), the optical detector(s) ( 6 ) and the single image/object plane ( 7 ) are located on the same optical axis ( 1 ). This simple, easy-to-use and compact apparatus can meet many different requirements and serve a large range of high throughput applications.

FIELD OF THE INVENTION

This invention relates to an optical system for four-dimensional (4D:3D+time) multi-plane broadband imaging which can transfer 4D wavefrontinformation between object and image spaces, i.e. simultaneouslycapturing multi-colour images from several object planes on a singleimage plane, or by an alternative implementation, simultaneouslyrecording chromatically-corrected images from a single object plane on afew image planes. This technique is versatile enough to combine withvarious modern techniques including microscopy, astronomical optics,optical data storage, biomedical imaging, wavefront analysis andvirtual/augmented reality, and will serve a large range of applicationsin academic research and industry.

BACKGROUND

The recent emergence of super-resolution microscopy imaging techniqueshas surpassed the diffraction limit to improve image resolution. Despitethe breakthroughs in spatial resolution, high temporal resolutionremains a challenge. Simultaneous multi-plane imaging has beenincreasingly explored, which opens the possibility for real time imagingof rapidly changing objects in cell-biology, fluid-flow problems andother high-speed, 3D tracking applications. In a technique originallydeveloped by Blanchard and Greenaway, a three-dimensional (3D) imagingsystem based on a diffractive optical element (DOE) in the form of anoff-axis Fresnel zone plate can be utilized to perform simultaneousthree-plane imaging using a simple, on axis optical set-up (A. H.Greenaway and P. M. Blanchard, ‘Three-dimensional imaging system’,International application published under the patent cooperation treaty(PCT), PCT/GB99/00658, (1999)). The DOE, which behaves like amulti-focus “lens” but utilizes the principle of diffraction instead ofrefraction, provides an order-dependent focussing power to generateseveral images. However, due to the inherent dispersion property ofnon-zero diffraction orders, this DOE based technique had to benarrow-band to limit the incident spectral bandwidth and thus chromaticdispersion, restricting photon flux and hindering application tomultiple-fluorophore life science imaging especially when theillumination source of samples was intrinsically faint. A few chromaticcorrection schemes were proposed which could in principle manipulate apolychromatic incident beam, output a pre-dispersed collimated beam andthus be used in combination with a DOE-based 3D broadband optical systemfor multi-colour imaging without sacrificing too much photon flux (P. M.Blanchard and A. H. Greenaway, ‘Broadband simultaneous multiplaneimaging’, Optics Communications 183(1), 29-36 (2000); Y. Feng, et al.‘Optical system’, UK Patent Application No. GB2504188-A, (2013)).However, due to the limitations in design of the key optical elementsand the imaging system, these schemes were only applicable to chromaticcorrection tests; not real 3D multi-plane broadband imaging. Thereforein order to make this technique available for 4D multi-plane broadbandimaging, I have invented an approach to analytically design the keyoptical elements and set up the imaging system with well-matchedparameters of a grating-grism combination, such that the previousoptical apparatus can be effectively improved and be versatile enough tocombine with various techniques (i.e. microscopy, astronomical optics,optical data storage, biomedical imaging, wavefront analysis andvirtual/augmented reality).

1. The Previous 3D Narrow-Band Imaging System and its Deficiencies

The original 3D narrow-band imaging system was comprised of a DOE, alens system (with a single or multiple lenses), a narrow bandpass filterand some apparatus for imaging (i.e. light source and camera). FIG. 1 isa schematic of the basic design of this 3D narrow-band imaging system.QD (quadratically distorted) grating, which is a form of DOE, allowsmultiple object planes with a separation of Δz to be simultaneouslyimaged on a single image plane. It is put in the telecentric positionfor equal magnification of the images (P. A. Dalgarno, et al.‘Multiplane imaging and three dimensional nanoscale particle tracking inbiological microscopy’, Optics Express 18(2), 877-884 (2010)).

f_(g), the equivalent focal length of diffraction order m of a QDgrating, can be expressed as,

$\begin{matrix}{f_{g} = \frac{R^{2}}{2\; {mW}_{20}}} & (1)\end{matrix}$

where R is the radius of the QD grating aperture and W₂₀ is the standardcoefficient of defocus. The focal lengths f_(g) are equal and oppositein sign for any m diffraction orders.

The separation between object planes Δz can be written as,

$\begin{matrix}{{\Delta \; z} = \frac{f_{eff}^{2}}{f_{g}M^{2}}} & (2)\end{matrix}$

where f_(eff) is the effective focal length of the lens system and M isthe magnification of the microscope objective.

Therefore the image separation on camera plane Δd can be written as,

$\begin{matrix}{{\Delta \; d} = {f_{eff}\left\lbrack {\arcsin \left( \frac{m\; \lambda}{d_{0}} \right)} \right\rbrack}} & (3)\end{matrix}$

where λ is the incident wavelength and d₀ is the central period of theQD grating.

In principle, the maximum dispersion across the QD grating s_(g) can begiven by,

$\begin{matrix}{s_{g} = \frac{{\Delta\lambda}\; R^{2}}{2W_{20}d_{0}}} & (4)\end{matrix}$

where Δλ is the bandwidth of incident spectrum.

Due to the practical requirements of broadband imaging and reasonablephoton flux, a dispersion device was implemented in the 3D imagingsystem to correct the chromatic distortion of non-zero diffractionorders induced by QD grating and reduce the energy loss. P. M. Blanchardand A. H. Greenaway demonstrated a chromatic correction scheme whichused a pair of reflective blazed gratings and a folded optical path tocompensate for the chromatic distortion by introducing an opposingchromatic shear, as shown in FIG. 2 (P. M. Blanchard and A. H.Greenaway, ‘Broadband simultaneous multiplane imaging’, OpticsCommunications 183(1), 29-36 (2000)). The amount of chromatic shear wascontrolled by changing the distance between the blazed gratings but,because of the folded path, changing the grating separation necessitatesadjustments of the angle and/or position of various optical components.Adjusting these additional parameters complicates the system, making itharder to integrate into user instrumentation and restricting practicalapplication.

Y. Feng et al. developed another chromatic correction scheme using apair of grisms (a combination of blazed grating and prism), but thisprevious imaging set-up cannot simultaneously capture three-plane imagesand was not compatible with any microscope because of the limitations ofthe design of key optical elements and mismatched parameters of thegrating-grism combination (Y. Feng, et al. ‘Optical system’, UK PatentApplication No. GB2504188-A, (2013)). Therefore, this roughgrating-grism system was difficult to use in real world imagingapplications, see section “QD grating-grism combination” for details.

2. Basic Principles of the DOE (QD Grating)

The DOE (“QD grating” in this invention) consists of a set of concentricarcs that alternate between either opaque/transparent (with regards toamplitude) or different optical thickness (with regards to phase). Withconsideration to a single-etch (two-level or say binary) QD grating, wedefine an X-Y Cartesian coordinate system as shown in FIG. 3, in whichthe origin is the geometric centre of the QD grating's mask pattern, thex-axis is perpendicular to the grooves, and the y-axis is parallel tothe grooves in the QD grating. We note that the integer values of n arethe loci number of each QD grating arc−n=0 corresponds to an arc thatpasses through the origin and the values of n vary from positive tonegative which are opposite in sign to the direction of x-axis.

The equation of the QD grating arcs is,

$\begin{matrix}{{\frac{x}{d_{0}} + \frac{W_{20}\left( {x^{2} + y^{2}} \right)}{\lambda \; R^{2}}} = n} & (5)\end{matrix}$

where x and y are Cartesian coordinates relative to an origin on theoptical axis in the plane of the QD grating, d₀ is the central period ofthe QD grating, W₂₀ is the standard coefficient of defocus, λ is theincident wavelength and R is the radius of the QD grating aperture whichis centred on the optical axis. Please note that a circular aperture isassumed in equation (5), but an aperture of any shape can be utilized.

Thus the radii of the nth concentric QD grating arc C_(n) can be writtenas,

$\begin{matrix}{C_{n} = \left\lbrack {\frac{n\; \lambda \; R^{2}}{W_{20}} + \left( \frac{{\lambda \;}^{2}R}{2d_{0}W_{20}} \right)^{2}} \right\rbrack^{1/2}} & (6)\end{matrix}$

Since the period of the QD grating across the aperture is not constant,this period d at a distance x from the origin along the x-axis can begiven by,

$\begin{matrix}{d = \frac{d_{0}\lambda \; R^{2}}{{\lambda \; R^{2}} - {2d_{0}W_{20}x}}} & (7)\end{matrix}$

Therefore with x=−R the minimum QD grating period d_(min) is,

$\begin{matrix}{d_{\min} = \frac{d_{0}\lambda \; R}{{\lambda \; R} + {2d_{0}W_{20}}}} & (8)\end{matrix}$

Equation (8) determines the accuracy of mask pattern in QD gratingfabrication.

Here an additional phase term to incident light, which is the so-calleddetour phase, can be produced by the quadratic distortions in adirection perpendicular to the QD grating grooves rather than the etchdepth (optical thickness) phase term produced by a normal phase grating.This local detour phase shift ϕ_(m) imposed on the wave front diffractedinto order m can be given by,

$\begin{matrix}{{\varphi_{m}\left( {x,y} \right)} = {m\frac{2\pi \; W_{20}}{\lambda \; R^{2}}\left( {x^{2} + y^{2}} \right)}} & (9)\end{matrix}$

Equation (9) shows that the phase delay imposed on the wavefrontsscatters into the non-zero diffraction orders and thus the QD gratinghas focussing power in these orders.

3. QD Grating-Grism Combination

Since the diffraction angle which determines the separation of thediffraction orders is wavelength dependent, for each wavelength that isinput into the grating an image will be produced at a unique position.With broadband illumination, a series of monochromatic images withdifferent positions in the non-zero diffraction orders are introduced bypolychrome incident light, giving chromatically smeared images in theseorders. Based on the inherent non-periodic chirps of QD grating, eachincident wavelength can be manipulated to “see” an appropriate period ofQD grating and identical diffraction angles with respect to differentwavelengths can be obtained, and thus mitigating the chromaticdispersion. As demonstrated in FIG. 4, a customized optical system canpre-disperse the polychrome input beam and produce collimated,chromatically displaced output, such that each colour is positionedwithin the chirped zone plate to “see” the same QD grating structuremeasured in wavelength units (this pre-dispersion thus equalises thediffraction angle for each wavelength), correcting the chromaticdispersion.

Using a pair of grisms (a blazed grating combined with a prism), achromatic correction scheme was demonstrated by Y. Feng et al. (Y. Feng,et al. ‘Optical system’, UK Patent Application No. GB2504188-A, (2013)).To verify the feasibility of the scheme, two commercial off the shelf(COTS) components—a 18° 8′ wedge-angle prism fabricated from N-BK7 and aB270 transmission grating of 300 grooves/mm and a blaze angle of 17° 30′(both from Edmund Optics), were selected and then cemented together.Although the cementing process was not accurate enough such that the twogrisms performed observable different jobs, an un-deviated wavelength of˜532 nm could be provided. See FIG. 5 for the schematic diagram of thebasic design of the multi-colour three-plane imaging system, in whichthe two cemented grisms were mounted back to back along the optical axis(thus their gratings are opposite to, and face each other) within anassembly using optomechanical components (Thorlabs). The centralwavelength of ˜532 nm passed through the grism pair un-deviated, but theshort and long wavelengths of the polychromatic beam were dispersed bythe first grism and collimated by the second one.

However, besides the chromatic dispersion, the imaging quality of theaforementioned optical system is also affected by the blurredoverlapping images produced by the re-entrant grooves of QD grating andthe residual dependence of the focal length of QD grating. Here the“re-entrant grooves” represent that a greater than hemi-circular patternof the Fresnel zone plate grooves appears in the mask pattern of the QDgrating. Due to the use of non-analytically designed QD grating withre-entrant grooves (as FIG. 6 shown), the image was of poor qualityalthough the chromatic dispersion can be effectively diminished by thegrism pair (Y. Feng, et al. ‘Optical system’, UK Patent Application No.GB2504188-A, (2013)). Moreover, the grism system was also lacking intheoretical design, such that the optical parameters of QD grating andgrism system were mismatched. Accordingly the optical path of thisimaging set-up was so long (the overall optical length was ˜1.3 metres,see FIG. 7) that the camera could only record one image corresponding toa single diffraction order each time, which could not even achieve theoriginal goal of simultaneous multi-plane/multi-order imaging. Inaddition, this imaging system is incompatible with other techniques,i.e. microscopy, because of various errors including inaccuratealignment of the long and unstable optical path, and the defects in thefabrication of grisms and their mounts.

DESCRIPTION OF THE INVENTION

In this invention, analytically-designed QD gratings without re-entrantgrooves are utilized to improve image quality. At the same time, aMathematica ray-path model of a grism system is established and thus theparameters of the grism can be theoretically customized. With thewell-designed grating-grism system, our 4D multi-plane broadband imagingapparatus can be successfully combined with other techniques, i.e.microscopy, for high quality imaging.

In principle, the multiple images in an image plane perpendicular to theoptical axis correspond to the diffraction orders scattered from the QDgrating structure. Consequently, it is important to delicately designthe QD grating structure to meet the requirements of practicalapplications. To utilize this type of QD grating in our 4D multi-planeimaging system, we concentrate more on the mask pattern design of the QDgrating structure in addition to the basic design developed by Blanchardand Greenaway. According to equations (6) and (7), the period dmaximises and the radius C_(n) minimises at the limit when x=R, whichmeans that C_(n) may become negative and d may be comparatively large.Under this circumstance, one or more grooves of the QD grating willbecome more than a half circle (a whole concentric circle may evenoccur), which is regarded as the “reentry phenomenon” (as FIG. 6 shown).This reentry phenomenon might be beneficial for some applications butmust be avoided in this invention. Overlapping images will be introducedby the re-entrant grooves, and the pre-dispersion produced by the grismsmay not fit the grating structure, and thus the performance of our 4Dimaging system will be degraded or even ruined. Here the “re-entrantgrooves” represent that a greater than hemi-circular pattern of theFresnel zone plate grooves appears in the mask pattern of the QDgrating. To specialize the DOE applied in our optical system, we definethe QD grating without re-entrant grooves as the non-reentryquadratically distorted (NRQD) grating.

By introducing the analytically-designed NRQD grating and grism, thisinvention develops a scanless 4D multi-plane broadband imaging system toimprove temporal resolution without compromising spatial resolution.This technique can transfer 4D wavefront information between object andimage spaces, i.e. simultaneously capturing multi-colour images fromseveral object planes onto a single image plane, or by an alternativeimplementation, simultaneously recording chromatically-corrected imagesfrom a single object plane onto a few image planes. This 4D multi-planebroadband imaging system comprises various optical components as follows(FIG. 8):

one or more NRQD gratings, which are defined as QD gratings withoutre-entrant grooves, arranged in a multi-element optical system toproduce a focal length and a spatial position associated with eachdiffraction order;

one or more pairs of grisms arranged to manipulate the optical path inspace by wavelength for correcting the chromatic dispersion of abroadband input beam generated by the NRQD grating(s);

a lens system arranged to effectively modify the focal length of theoptical system associated with each diffraction order of the NRQDgrating(s) and manipulate the optical path to meet the designrequirements of the grism system and,

means for light detection.

The NRQD grating can be designed by a combination mask pattern whichcomprises more than one NRQD arc pattern such that the in-focusmulti-plane (more than 3) images can be spatially arranged on a singleimage plane.

To obtain higher optical efficiency, NRQD gratings can be finelyfabricated to achieve a multi-level (digitised) or continuous-level(analogue) profile structure.

A variety of NRQD grating types can be utilized, which consist of, butare not limited to, alternate regularly spaced grooves of differenttransmissivity, reflectivity, optical thickness or polarisationsensitivity.

The grisms applied in this invention can be volume phase holographic(VPH) grisms.

A grism, a blazed grating and prism combination, is analyticallydesigned by a Mathematica ray-path model for chromatic correction of the4D multi-plane broadband imaging system. The design of a grism can bedefined by its groove density, which can be specified from 100 to 800lines per mm when the refractive index of the grism substrate is1.4-1.5; from 100 to 900 lines per mm when the refractive index of thegrism substrate is 1.5-1.6; from 100 to 1200 lines per mm when therefractive index of the grism substrate is 1.6-1.7; and from 100 to 1400lines per mm when the refractive index of the grism substrate is greaterthan 1.7.

The refractive indices of the grism components, blazed/VPH grating andprism(s), can be different.

More than one pair of grisms can be utilized such that the chromaticdispersion of a broadband input beam generated by more than one NRQDgrating can be corrected.

The grisms can be located at any plane as long as the theoreticalpre-dispersion and re-collimation for the full incident waveband can befulfilled.

In this invention, our 4D multi-plane broadband imaging system iscompatible with multi-mode commercial microscopes includingfluorescence, bright/dark field, phase contrast, differentialinterference contrast (DIC) and structured illumination.

By focussing a single broadband illumination source on a series ofdifferent planes, this system can also be utilized for multi-planebroadband illumination.

This technique is versatile enough to combine with various moderntechniques including microscopy, astronomical optics, optical datastorage, biomedical imaging, wavefront analysis and virtual/augmentedreality, and will serve a large range of applications in academicresearch and industry.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the schematic of the basic design of the 3D narrow-bandimaging system.

FIG. 2 shows the schematic diagram of an early chromatic correctionscheme for pre-dispersion of light before it is incident on the QDgrating.

FIG. 3 shows a demonstration of a QD grating structure in an x-yCartesian coordinate system.

FIG. 4 shows the pre-dispersion and collimation of incident broadbandlight before the QD grating can correct the chromatic dispersion.

FIG. 5 shows the schematic of the basic design of a multi-colour imagingsystem based on a QD grating and a grism pair. As described in patentGB2504188-A, the imaging system (40) comprises a dispersive device (10)optically aligned between a pair of achromatic lenses (42), (44), apolychromatic light source (32), a QD grating (46), a multimode fibre(34) and a CCD detector (not shown).

FIG. 6 shows the re-entrant grooves in a partial view of the maskpattern of the QD grating used in the UK patent GB2504188.

FIG. 7 shows the long optical path of a QD grating and grisms basedimaging system.

FIG. 8 shows the schematic of the four-dimensional (4D) multi-planebroadband imaging system. The imaging system comprises: one or morenon-reentry quadratically distorted (NRQD) gratings (5) which canproduce a focal length and a spatial position corresponding to eachdiffraction order, thus simultaneously transmitting wavefrontinformation between multiple object/image planes (2) and a singleimage/object plane (7); a grism system (4) which can limitchromatically-induced lateral smearing by creating a collimated beam inwhich the spectral components are laterally displaced; a lens system (3)which is configured to adjust the optical path; and the opticaldetector(s) (6). In an optical system, the multiple object/image planes(2), the lens system (3), the grism system (4), the NRQD grating(s) (5),the optical detector(s) (6) and the single image/object plane (7) arelocated on the same optical axis (1).

FIG. 9 shows the ray-path model of a single grism, showing ray pathsfrom air to grism, and back to air (the blazed order is +1 in thiscase).

FIG. 10 shows that, for a substrate of B270 (SCHOTT) glass, thewedge/blazed angle of a grism can be customized at a specificun-deviated wavelength. The legends mark the number of grooves of blazedgratings (lines/mm), corresponding to the joint lines from bottom to topin the graphs, respectively.

FIG. 11 shows “Yan's rainbow”, which is the first time for thisinvention's imaging technique to successfully record the 4D broadbandimages of all the three diffraction orders in a single snapshot. This isaccomplished without the degradation of image quality caused by the NRQDgrating-induced λ dependent focussing power.

FIG. 12 shows the false colour images from a bandpass-filtered whitelight source measured: (i) without chromatic correction by grisms. Inthis figure there is intense blurring of the image spots, whichcorresponds to non-central wavelengths, and is caused by the dispersionand the difference in focal lengths between wavelengths induced by theQD grating. The re-entrant grooves of the QD grating also cause a kindof blurring resulting from overlapping images. (ii) with chromaticcorrection by grisms. The dispersion has been effectively mitigated, butthe image spots corresponding to non-central wavelengths are stillblurry. This blurring is generated by the difference in focal lengthsbetween wavelengths induced by the QD grating as well as the re-entrantgrooves of the QD grating. (iii) at optimal foci for each colour withoutchromatic correction by grisms.

FIG. 13 shows the 4D three-plane broadband images of eGFP fluorophorewith and without chromatic correction by grisms, in which white lightand a series of bandpass filters were combined to simulate microscopyimaging.

FIG. 14 shows the 4D three-plane broadband microscopy images offluorescence microspheres with and without chromatic correction bygrisms.

FIG. 15 shows the bright field 4D three-plane broadband microscopyimages of HeLa living cells. The separation between each in-focus image(Δz, see equation (2)) is 2.3 μm and the bandwidth is 525±39 nm.

FIG. 16 shows that, by combining the phase contrast imaging mode withmulti-plane imaging principles, an NRQD grating with this type of maskpattern can be utilized for 4D three-plane broadband phase contrastmicroscopy imaging.

FIG. 17 shows that, for a substrate of fused silica, the wedge/blazedangle of a grism can be customized at a specific un-deviated wavelength.The legends mark the number of grooves of blazed gratings (lines/mm),corresponding to the joint lines from bottom to top in the graphs,respectively.

FIG. 18 shows that, for a substrate of N-BAF10 (SCHOTT) glass, thewedge/blazed angle of a grism can be customized at a specificun-deviated wavelength. The legends mark the number of grooves of blazedgratings (lines/mm), corresponding to the joint lines from bottom to topin the graphs, respectively.

FIG. 19 shows the schematic of the mask pattern of an NRQD grating with“crossed” structure, by which the images of nine separate object planescan be simultaneously obtained and split across a single image plane.

FIG. 20 shows that, for a substrate of N-SF11 (SCHOTT) glass, thewedge/blazed angle of a grism can be customized at a specificun-deviated wavelength. The legends mark the number of grooves of blazedgratings (lines/mm), corresponding to the joint lines from bottom to topin the graphs, respectively.

IMPLEMENTATION DETAILS AND EXAMPLES

The main challenges of this invention are the analytical design of theNRQD grating and the grism, and the well-matched parameters of an NRQDgrating-grism combination. In practice, to avoid re-entrant grooves andthus overlapping images, we must set appropriate values of the centralperiod d₀ and the defocus level W₂₀. Here the central period determinesthe diffraction angle and therefore the separation of images recorded onthe image detector (i.e. camera). Then a lens system is taken intoaccount, accompanied with other parameters of the NRQD grating, i.e.incident waveband, radius, refractive index of substrate and etch depth,with which a relay design of NRQD grating-lens combination can beroughly established. Based on this grating-lens combination model, theparameters of the grism are carefully considered, such that the functionof chromatic control can be fulfilled and the relay design of an NRQDgrating-lens-grism combination can be optimized. Finally, to obtain an“optimal” relay design, a few rounds of parameter optimization arealways performed. During this process, re-entrant grooves may occur andthe minimum period d_(min) (equation (8)) of the NRQD grating may becometoo small, resulting in either a flaw in the design of the NRQD gratingor higher complexity of grating fabrication. Therefore, it is essentialto perform a final check on the two factors of d₀ and W₂₀ beforecarrying out simulation experiments with the “optimal” parameters. Wenote that, applying similar principles, variations of the so-called“optimal” relay design are always available in practice.

Based on the theories of Fraunhofer diffraction and Fourier optics, a 2Dmathematical model of the NRQD grating is established, by which we coulddeeply explore the imaging principles of NRQD grating as well as improvethe design of NRQD grating (i.e. design of mask pattern and fabricationparameters including an optimal etch depth). Following the Zernikepolynomial, a complicated Matlab program was developed for plotting themask pattern of a QD grating. However, the plotting process was opaque.This opacity, given the complexity of the algorithm and itscorresponding long run time (normally more than 10 hours), means thatthe parameters we start the plotting with may be inappropriate such thatwe have to check for re-entrant grooves after the calculations arecompleted. These shortcomings make tuning the grating design parametersdifficult, reducing this algorithm's flexibility with regards to gratingdesign. This algorithm also has the limitation that only “arc” grooves,rather than any other combined structures such as “crossed” grooves, canbe generated. This inability to generate crossed grooves hinders thedevelopment of improved grating designs. In this invention, based on our2D mathematical model of NRQD grating, we have developed codes usingboth Mathematica and Matlab software for optimizing the parameters ofplotting the mask pattern. Then with these optimal parameters, aplotting program developed by AutoCAD software is applied to “draw” themask pattern of the NRQD grating, such that the defects of the masklayout may be easily found during the visible plotting procedure. Aswell, the processing time of the mask-generation is significantlyreduced; only a few minutes for an experienced AutoCAD user.Furthermore, the mask layout of either a single NRQD grating or acombination of two or more NRQD gratings can be flexibly generated andmanipulated to meet further imaging requirements. It is known that themore real gratings introduced, the more energy lost by the opticalsystem. In comparison with overlaying two or more NRQD gratings uponeach other for simultaneous 9- or more-plane imaging, a single masklayout with an arbitrary combination pattern can be produced andtherefore 9 or more images can be efficiently obtained uponimplementation of this invention. In any case, the chromatic dispersioninduced by more than one NRQD grating or a single NRQD grating with itscombination mask pattern, can be corrected by more than one pair ofgrisms.

Our ray-path model of a single grism is based on Traub's design (W. A.Traub, ‘Constant-dispersion grism spectrometer for channeled spectra’,Journal of the Optical Society of America A 7(9), 1779-1791 (1990)). AsFIG. 9 illustrated, a ray indicated by the arrow lines successivelyenters the prism at angle A, refracts, reaches the grating at angle α,and finally diffracts into the air at angle β (+1st order here), wherethe dashed lines represent the directions of the prism normal (PN), thegrating normal of the outer face (GN) and the facet normal (FN). Thesign convention is that, measuring from the normal to the incidentsurface, counterclockwise angles are positive. Such a grism can beobtained by manufacturing a grating structure onto one face of a prism.At this stage, we only concentrate on a simple and practical grismdesign, rather than demonstrating the full descriptions of themathematical ray-tracing model.

For the grism model shown in FIG. 9, the diffraction angle β can begiven by,

$\begin{matrix}{\beta = {\arcsin \left( {{n\; {\sin \left( {{\arcsin \left( \frac{\sin \mspace{14mu} A}{n} \right)} + E} \right)}} - \frac{m\; \lambda}{d}} \right)}} & (10)\end{matrix}$

where m is the number of the diffraction order, λ is the incidentwavelength, d is the grating period, and n is the refractive index ofthe grism substrate. The positive orders diffract clockwise with respectto the zeroth order, whereas the negative orders diffractcounterclockwise. Please note that in equation (10) the refractive indexn is assumed to be the same for the grism components, a grating and aprism. However, these components can have different refractive indices.Using the grism ray-path model, the refractive index of each componentcan be easily changed.

Based on the ray-path model of the grism system, the parameters of thegrism such as the wedge/blazed angle and the number of grooves of theblazed grating can be calculated using the desired glass type of thegrism substrate and the un-deviated wavelength. We have a set ofpre-calculated tables of such parameters in our figures corresponding tocertain glass types and un-deviated wavelengths. The grism parameterscan be optimized according to the design of the NRQD grating and therequired properties of the optical system. A ray-tracing simulation inZemax software is applied to verify that the parameters of the NRQDgrating and grisms are well matched, and that the performance of theoptical system is optimized. Consequently, a customized 4D multi-planebroadband imaging system based on NRQD grating and grism is established.In principle, grisms utilized in an imaging system may be of anysuitable type without compulsorily identical design, and their positionsin optical relay system may be arbitrary as long as the amount ofpre-dispersion required for sufficient chromatic correction can beprovided.

In this invention, we have built a set of mathematical models based onMathematica and Matlab software for the design of key optical elementsand the NRQD grating-grism combination system. Under the guidance ofthese theoretical models, the optical system can be customized for avariety of applications and is versatile enough to combine with variousmodern techniques including microscopy, astronomical optics, opticaldata storage, biomedical imaging, wavefront analysis andvirtual/augmented reality. Here we will demonstrate some practicalexamples for 4D multi-plane broadband imaging, which, by an alternativeimplementation, can also be utilized in capturing broadbandillumination/images from a single object plane on a few image planes. Askilled person will appreciate that variations of the disclosedarrangements are possible without departing from the invention.Accordingly, the following description of the specific embodiments ismade by way of example only and not for the purposes of limitation. Itwill be clear to the skilled person that minor modifications may be madewithout significant changes to the operation described.

Example 1: Simulated 4D Three-Plane Broadband Imaging Tests

We have demonstrated the principles and the design of the 4D multi-planebroadband imaging system. To test the quality and effectiveness of thisapproach for practical use, some simulated 4D three-plane broadbandimaging experiments were performed, such that both our theoreticalmodels and optical set-up can be qualitatively verified.

A continuum white laser (Fianium supercontinuum light sourceSC450-PP-HE, operating wavelength ranges from approx. 450 nm to >1750nm) fed through a single mode optical fibre (Thorlabs P1-488PM) wasutilized to mimic imaging environments. Here a simple optical set-up weused before (FIG. 5) was built. A pinhole with a diameter ofapproximately 3 mm was used as an aperture stop, and two achromats withan identical focal length of 250 mm were spaced 200 mm apart andoperated as a unit magnification relay system with an effective focallength of ˜208 mm. In this case the NRQD grating, which is fabricated onfused-silica substrate with a refractive index of ˜1.46, has a nominalaxial period (i.e. central period) of 50 μm, a curvature W₂₀ of 50 wavesand radius 10 mm, thus has ±1898 mm focal lengths in the firstdiffraction orders at 527 nm incident wavelength. When the NRQD gratingwas placed 208 mm from the second principal plane (˜42 mm from thesecond achromat) of the compound imaging system, equal magnificationimages were obtained in each diffraction order, providing simultaneousthree-plane imaging due to the order dependent focussing powerintroduced by the NRQD grating.

A ray-path model, in which a pair of identical, back to back grisms(their gratings are opposite to, and face each other) are applied, hasbeen established for analysing the selection of grism parameters.According to this ray-path model, the customized parameters of the grismcan be described as a set of tabulated functions, from which the wedgeand blazed angles and the number of grooves of the blazed grating forcertain optical glass at a specific un-deviated wavelength, can beestimated/selected based on the practical requirements of the opticalsystem. In this case, the customized grism is made from Schott B270substrate with a refractive index of ˜1.53 and has a squared size of 25mm×25 mm. Based on the tabulated functions shown in FIG. 10, when theun-deviated wavelength of the grism is 527 nm, both the wedge and blazedangles (corresponding to E and E′ in FIG. 9, respectively) of 17.5° and300 grooves/mm for the blazed grating can be selected. Here the gratingstructure can be fabricated onto the hypotenuse face of a right-angleprism, and the wedge angle of the prism and the blazed angle of thegrating are identical. In a simple optical system similar to the oneshown in FIG. 5, our customized grism pair can produce a collimated beamwith chromatic shear from a collimated polychrome input, and the lateralshear between the polychrome components in the output beam can becontrolled by varying the separation between the grisms. Note that onlythe spacing between the two grisms is relevant, not their absoluteposition between the achromat pair.

Before performing our imaging tests, we need to define an appropriatephysical separation of the images on the image plane (the sensor chip ofthe camera) to avoid the multi-plane images overlapping with each other,and also make good use of the sensor chip. Due to the physical size ofthe camera sensor chip, we need to arrange the multi-plane (2˜9) imagesto fit onto this chip. In this case, the sensor chip of our camera has a2048×2048 array of 6.5 μm square pixels (Andor Zyla 4.2 sCMOS),therefore the size is 13.3 mm×13.3 mm. For telecentric imaging (1:1magnification of the optical system) we built a simultaneous three-planebroadband imaging system by using the selected parameters shown above.At normal incidence, when the design wavelength is 527 nm and thecentral period of the NRQD grating is 50 μm, the diffraction angle ofthe first orders is about 0.6°. Accordingly, a centre separation of 2.18mm between the three images (0th and ±1st orders) is achieved whichmight be regarded as the minimum separation between each image withoutoverlapping. Thus 3×2.18 mm=6.54 mm (˜50%) of the camera chip width willbe occupied by these images, which presents a well-designed opticalsystem.

To assess the performance of the 4D multi-plane broadband imaging systemover the entire visible spectrum, a high-power fibre continuum source(Fianium SC450-PP-HE) filtered by a set of 20 nm bandpass filters(Thorlabs) with central wavelengths from 450 nm to 650 nm in 20 nm stepswas implemented. Thus eleven wavebands, each with a 20 nm bandwidth, canbe simulated. The different object distances corresponding to the threeobject planes were achieved by varying the fibre light source. Eachgreyscale image captured by the sCMOS camera (Andor Zyla 4.2) wasnormalized for equal total photon flux at each waveband using ImageJsoftware, and then falsely coloured using Mathematica software by meansof calculating the RGB value of each central wavelength, normalizingeach image for compatibility with RGB, creating coloured images using R,G and B scalers, and compositing a coloured image of a single centralwavelength. Finally, the false-colour images of all the eleven wavebands(central wavelengths from 450 nm to 650 nm in 20 nm steps) were combinedtogether and thus formed a multi-colour image. Since this compositeimage presents an impressive phenomenon of rainbow hues, the inventor YFlabels it following her name—Yan's rainbow, which is supposed to specifythe multi-plane, multi-colour images introduced by the NRQD grating withand without grism correction. The “rainbow” images in greyscale formatare shown in FIG. 11.

Yan's rainbow demonstrates the first successful use of our 4D imagingapproach based on NRQD grating and grism for simultaneous multi-planebroadband imaging. In comparison with the similar optical design YF etal. proposed in 2013 (Y. Feng, et al. ‘Optical system’, UK PatentApplication No. GB2504188-A, (2013)), a few advantages of ourwell-designed optical elements as well as the imaging system have beenrevealed by Yan's rainbow. First, our current system can be implementedfor simultaneous three-plane (in this case) imaging with chromaticcorrection, whereas the old system YF et al. developed in 2013 can onlyrecord a single-plane image corresponding to one of the threediffraction orders in each snapshot (see FIG. 12). Second, our systemhas successfully mitigated the smearing of non-central wavebands (wherethe central waveband is 520 nm to 540 nm) induced by the dependentfocussing power of the NRQD grating through the appropriate parametersof NRQD grating and grisms, rather than compensating for the focaldependence by re-focussing the camera for each chromatic component as YFet al. did before (FIG. 12 (iii)). Without repositioning the fibre lightsource at each waveband to compensate for the NRQD grating induced focaldependence, the imaging system we built in this case was only focussedfor the un-deviated/central wavelength of ˜530 nm in the first threediffraction orders, with and without grism correction. It is evidentthat, in Yan's rainbow, the image quality of the non-zero diffractionorders of each non-central waveband is effectively improved, whereas thezeroth order image is unaffected by NRQD grating and grisms (except thelower photon flux). Furthermore, although the chromatic dispersion ofnon-central wavebands has been mitigated, the spots shown in FIG. 12(ii) are still smeared. Unlike Yan's rainbow, the image quality in thisfigure is affected by the blurred overlapping images produced by the QDgrating's re-entrant grooves (see FIG. 6) and the residual λ dependenceof the focal length. And it is not practical to adjust for the linearlywavelength dependent focussing power of the QD grating by repositioningthe light source at optimal foci for each colour as illustrated in FIG.12 (iii). Consequently, to achieve effective 4D multi-plane broadbandimaging, the appropriate QD grating(s) without re-entrant grooves (thusNRQD grating(s)) is(are) essential, and a well-designed optical systembased on NRQD grating(s), grisms and achromatic doublets should bemathematically modelled, verified, and optimized.

We note that, since the rainbow experiment was performed to simulate thefunction of 4D multi-plane broadband imaging technique, a relativelysmall grism separation of 140 mm was selected in this case, by whichchromatic correction for only a ˜100 nm bandwidth could be completedaccording to the mathematical model (here the incident bandwidth is 220nm). Hence a residual chromatic dispersion of the first orders stilloccurred, as shown in FIG. 11. In fact, chromatic correction for abroader waveband can be achieved by enlarging the grism separation. Amore delicate mathematical model of 4D multi-plane broadband imagingsystem is in progress, which takes more optical aberrations intoaccount, i.e. spherical aberration and coma.

To further demonstrate the effectiveness of our 4D multi-plane broadbandimaging system, we took a simulation test of fluorescence microscopyimaging. As an illustration we have chosen to model eGFP fluorophore,which is widely used in cell biology. Without involving any microscope,we simulated the 4D multi-plane broadband microscopy imaging offluorophore by modifying the bandpass filtered white light (i.e. rainbowexperiment for certain waveband) and simply summing a series of imagesspanning the fluorophore emission bandwidth (from 480 nm to 600 nm in 20nm steps), with an appropriate weighting to mimic the fluorophoreemission spectrum. The optical system was focussed and aligned for thefilter that best matched the peak fluorophore wavelength of 520 nm. Dueto the optimization of the optical system and the lack of blurringproduced by the QD grating's re-entrant grooves, we may ignore thedifference in focal length between wavelengths induced by our NRQDgrating. With regards to that fact, we chose to fix the source positionfor the fluorophore, and only the zeroth order image was adjusted infocus. A series of 20 nm bandwidth images were then recorded using a setof spectral filters covering the fluorophore emission spectrum, and thespots corresponding to the individual filters (wavebands) were clearlyvisible. To process these narrow band images, the total flux for eachfilter (waveband) was first normalized to the total flux at the peakemission wavelength and then weighted to simulate the fluorophoreemission spectrum by the appropriate factor. The composite three-planeimages for the simulated eGFP fluorophore, with and without grism inplace, are shown in FIG. 13. We see that the three-plane broadband imageof the simulated eGFP fluorophore can be simultaneously recorded,although a minor residual chromatic dispersion occurs in the redwavebands (the top image in FIG. 13).

We have successfully developed the 4D multi-plane broadband imagingsystem based on NRQD grating and grism, which, for the first time,achieves chromatically-corrected, high-efficiency, easy-to-use, andsimultaneous three-plane imaging. Due to the high temporal resolution,this optical system could be applied to measure some dynamic procedures,such as single particle tracking. The simulated tests, which areperformed in the same condition as practical imaging (i.e. same incidentwaveband and photon energy distribution), may be taken as a referencefor reconstructing the accurate 4D images from multi-plane images suchthat the distortion of non-zero order images can be corrected bypost-image processing techniques.

Example 2: 4D Three-Plane Broadband Multi-Mode Microscopy Imaging

Although some modern optical microscopy techniques have achievedconsiderably high spatial resolution, specimen information can only beobtained from a single focal plane in each snapshot which is2-dimensional (2D). Biological samples, i.e. living cells, arethree-dimensional (3D) and constantly changing, so the observation of 3Dbiological specimens and thus the analysis of volume structures havebeen increasingly needed for basic biological research as well asclinical diagnosis and therapy. Most of the current 3D microscopyimaging techniques use time-consuming methods, such as scanning thedepth of a sample, leading to severe limitations for imaging opticallysensitive samples and exploring biosamples dynamics, especially whenfast dynamical processes are required to be followed. And unfortunately,the spatial and temporal resolutions are mutually opposed to oneanother, and the temporal resolution is always sacrificed for seeingfine structural details. Therefore, new temporal resolution imagingtechniques are required in order to record 3D dynamics in high temporalframe rates without compromising spatial resolution.

In this invention, we have built a high temporal resolution, highefficiency and easy-to-use 4D multi-plane broadband imaging system,which is versatile enough to combine with various modern techniquesincluding microscopy, astronomical optics, optical data storage,biomedical imaging, wavefront analysis and virtual/augmented reality.Here the efficiency of the grating is defined as the energy flow oflight diffracted into the orders being measured, relative to the energyflow of the incident light. The efficiency of an NRQD grating can beoptimized by high-precision fabrication (i.e. multi-etch), by which amulti-level (digitised) or continuous-level (analogue) profile structurecan be obtained. According to the various requirements of imagingapplications, a variety of NRQD grating types can be utilized, whichconsist of alternate regularly spaced grooves of differenttransmissivity, reflectivity, optical thickness or polarisationsensitivity. By using customized grisms with optimized parametersinstead of a narrow bandpass filter, the chromatic effects of an NRQDgrating can be efficiently controlled and the light flux of an imagingsystem can be significantly improved. In this section, we willdemonstrate a few practical applications in 4D multi-plane broadbandmicroscopy imaging by using our optical set-up. The imaging apparatuscan be easily appended to the camera port of a commercial microscope tosimultaneously record 4D multi-colour images from several object planesand can be used in various imaging modes, e.g. fluorescence,bright-field, phase-contrast, differential interference contrast (DIC),structured illumination, etc. Without the need of a narrow bandpassfilter and complicated adjustment of the optical system, our 4Dmulti-plane broadband microscopy imaging system is well suited tobiological applications, in which there is always a very limited amountof light available for imaging and the object to be measured constantlychanges. It is also compatible both with particle localization andtracking and with full-field, 3D, deconvolution-based specimenreconstructions from z-stacks. Z-plane separations of multi-plane imagescan be varied from arbitrarily small to many microns.

4D Three-Plane Broadband Microscopy Imaging of Fluorescence Microspheres

Since the full bandwidth of the emission spectrum of fluorophores can beaccessed using 4D multi-plane broadband imaging technique, 4Dmulti-plane fluorescence microscopy imaging becomes an importantapplication of this technique. Combining our 4D multi-plane broadbandimaging technique with microscopy, a simple and compact opticalapparatus has been built and appended between microscope and camera,tracking and recording the 4D optical information of specimens. Todemonstrate the optical performance of this apparatus, a 4D three-planebroadband microscopy imaging test of fluorescence microspheres wasperformed.

First, a fluorescence microspheres sample for the imaging test waswell-prepared. The coverslips (BRAND, 470820) were treated with acetoneand 1M NaOH solution successively, in which each treatment was performedin an ultrasonicator for 30 min, followed by thoroughly rinsing withdeionized water for several times (>2, sonicate when necessary). Theseclean coverslips were then dried with nitrogen. And due to the goodviscosity and optical properties, Polyvinyl Acetate (PVA, 81381Sigma-Aldrich) was chosen as the carrier material of fluorescencemicrospheres. 30% PVA aqueous solution was prepared by dissolving thepowder in water heated to about 100° C. under stirring. Then the 1:10diluted suspension of fluorescence microspheres (Invitrogen, F8827, 2μm, 505/515) was mixed with PVA solution with a ratio of 1:10, alongwith repeated sonicate, vortex mixer and 70° C. water bath which avoidedbeads clustering and allowed sufficient mixing of beads and PVAsolution. Finally, the well-dispersed fluorescence microspheres with PVAsolution were gently dipped onto the cleaned coverslips (˜100 μl persample, spin coating may be applied if a film with even thickness wasrequired) and left in a 45° C. oven for a few minutes for drying. With aquick check by microscope, we saw that the fluorescence microsphereswere monodispersed and quasi-stochastic uniformly immobilized in PVAgel. The samples were kept in dark place to avoid photolysis (althoughit rarely happened).

Imaging experiments were carried out on an Olympus IX73 microscopeset-up (100× oil-immersion objective) that was configured tosimultaneously image three object planes of the specimen onto a singleimage plane with chromatic correction. To fully demonstrate the emissionspectrum of fluorescence microspheres, our NRQD grating-grismcombination system was designed to give an un-deviated (central)wavelength at the peak of fluorophore's emission spectrum. Based on theoptical parameters applied in Example 1, the relay design of this 4Dthree-plane broadband fluorescence microscopy imaging system wasestablished, with the separation between the three object planes (Δz) of2.3 μm. This was achieved by placing the NRQD grating at Fourier planeof the lens system, but the absolute positions of the grisms were notstrictly defined. The sample of fluorescence microspheres was excited bya laser source with a wavelength of 473 nm, and a bandpass filter(Thorlabs, FB550-40) was implemented on the emission light and generateda spectrum bandwidth (Δλ) of 80 nm, which corresponded to a grismseparation of 108 mm for chromatic correction according to ourmathematical model. The images were captured by the sCMOS camera (AndorZyla 4.2) with an exposure time of 50 ms and then were processed by thesoftware ImageJ. In the images of the first three orders before andafter chromatic correction shown in FIG. 14, we see that the threeplanes of fluorescence microspheres can be simultaneously imaged in asingle snapshot and the chromatic dispersion of first diffraction ordershas been effectively corrected by the grism pair.

In this invention, our 4D multi-plane broadband microscopy imagingtechnique can also be applied for the simultaneous multi-plane imagingof multiple fluorophores. By using a set of NRQD gratings (each designedfor a different wavelength), grism pairs and dichroic mirrors,multi-plane in-focus images of multiple fluorophores can besimultaneously recorded on a few separate monochrome cameras. An examplewe presented before shows that (Y. Feng, et al.‘Chromatically-corrected, high-efficiency, multi-colour, multi-plane 3Dimaging’, Optics Express 20(18), 20705-20714 (2012)), if the dichroicmirror directs emission from a short wavelength emission fluorophore toone camera and the other camera sees a long wavelength fluorophore,these fluorophores can both be imaged separately but simultaneously in3D on the two cameras. After, a third fluorophore at a centralwavelength can be chosen such that the light from this fluorophore isdetected in 3D on both cameras. Then the coincident images on the twocameras are clearly due to the mid-wavelength fluorophore and, oncethese coincidences have been established, the remaining images on eachcamera can be assigned to the appropriate short or long wavelengthfluorophore. Similar to the principles above, if more than two NRQDgratings and more than two pairs of grisms are utilized, and if thetotal emission is split by a few dichroic mirrors, more emissionfluorophores with different wavebands can be simultaneously andmulti-plane imaged on a set of separate monochrome cameras. Since eachfluorophore is simultaneously and multi-plane imaged, it is possible tostudy dynamic interactions between different cellular components in 4Dwith accurate chromatic correction.

4D Three-Plane Broadband, Bright Field Microscopy Imaging of Living HeLaCells

HeLa cells were cultured in DMEM (Hyclone, U.S.A.) containing 10% fetalbovine serum (Hyclone, U.S.A.) in a humidified 5% CO₂ atmosphere at 37°C. The living cells, grown in a 35 mm glass-bottom dish (ShengyouBiotechnology), were then rinsed several times (normally 3 times) withPBS buffer and incubated in fresh medium for bright field microscopyimaging. The optical set-up remained almost the same as that of thesimulated experiments illustrated in Example 1, except replacing an NRQDgrating with a smaller central period of 30 μm and applying a grismseparation of 176 mm. In addition, an output spectrum bandwidth (Δλ) of78 nm was obtained by another bandpass filter (Thorlabs, MF525-39). Thenthe HeLa cells were illuminated by an unfiltered white light halogenlamp for three-plane bright field microscopy imaging using Olympus IX73microscope set-up (100× oil-immersion objective).

FIG. 15 shows the 4D three-plane broadband bright field microscopyimages of HeLa cells. The three images appear to be significantlydifferent and the shape of cells in non-zero order images looksundistorted. Since the separation between the three object planes Δz=2.3μm is so large that the imaging depth exceeds the axial size of thecell, these object planes should be regarded as in-focus although theimages are not sufficiently sharp. Further imaging experiment(s) will beperformed using a smaller separation (Δz) of ˜1 μm, which can capturemore details of inner structures of the cells.

4D Three-Plane Broadband Phase Contrast Microscopy Imaging

Our 4D multi-plane broadband imaging system provides a low-cost andflexible approach to the implementation of image capture involvingseveral different imaging modes, i.e. bright/dark-field, fluorescent,phase-contrast, differential interference contrast (DIC) and structuredillumination. In this case, a curved and partly dislocated NRQD gratingstructure is utilized to combine the 4D multi-plane broadband imagingtechnique with the phase contrast microscopy imaging mode, as FIG. 16shows (Y. Feng, et al. ‘Multi-mode microscopy using diffractive opticalelements’, Engineering Review 31(2), 133-139 (2011)). With adisplacement of one quarter grating period with reference to the outerNRQD grating structure, a phase shift of

$\frac{\pi}{2}$

for the +1st order and

$- \frac{\pi}{2}$

for the −1st order can be produced, which either retards or advances thephase of the diffracted reference beam, dependent on the diffractionorder (the zeroth order is unaffected).

In this case, the NRQD grating, which is fabricated on fused-silicasubstrate with a refractive index of ˜1.46, has a nominal axial periodof 32 μm, a curvature W₂₀ of 150 waves and a radius of 10 mm, and thushas ±538 mm focal lengths in the first diffraction orders at 620 nmcentral wavelength. Based on the tabulated functions (as shown in FIG.17) obtained by our ray-path model of the grism system, when thecustomized grism is made from fused silica substrate with a refractiveindex of ˜1.46 and the un-deviated wavelength of the grism is 620 nm,both the wedge and blazed angles (corresponding to E and E′ in FIG. 9,respectively) of 15.7° and the number of grooves of the blazed gratingof 200 grooves/mm, can be selected. Two achromats with an identicalfocal length of 300 mm were spaced 250 mm apart and operated as a unitmagnification relay system with an effective focal length of ˜257 mm. Abandpass filter (Thorlabs, MF620-52) was implemented to generate anoutput spectrum bandwidth (Δλ) of 104 nm, which corresponded to a grismseparation of 189 mm for chromatic correction. Based on the above relaydesign and applying a 100× oil-immersion objective, the separationbetween the three in-focus object planes (Δz) was 12.3 μm. This opticalsystem enables the 4D multi-plane broadband microscopy imaging/trackingof transparent and rapidly-moving objects (i.e. human sperm motilitymeasurement) in a considerable large volume, which may offer newinsights into biodynamics.

4D Three-Plane Broadband Differential Interference Contrast (DIC)Microscopy Imaging

In this case, our 4D multi-plane broadband imaging system is implementedfor the differential interference contrast (DIC) microscopy imagingmode. The imaging experiment can be performed on an Olympus IX73microscope set-up which is configured to simultaneously DIC image threeobject planes on to a single image plane with chromatic correction.

Here the NRQD grating, which is fabricated on fused-silica substratewith a refractive index of ˜1.46, has a nominal axial period of 30 μm, acurvature W₂₀ of 50 waves and a radius of 10 mm, and thus has ±2088 mmfocal lengths in the first diffraction orders at 479 nm centralwavelength. Based on the tabulated functions (as shown in FIG. 18)obtained by our ray-path model of the grism system, when the customizedgrism is made from N-BAF10 (SCHOTT) substrate with a refractive index of˜1.68 and the un-deviated wavelength of the grism is 479 nm, both thewedge and blazed angles (corresponding to E and E′ in FIG. 9,respectively) of 44.8° and the number of grooves of the blazed gratingof 1000 grooves/mm, can be selected. Two achromats with an identicalfocal length of 150 mm were spaced 130 mm apart and operated as a unitmagnification relay system with an effective focal length of ˜132 mm. Abandpass filter (Thorlabs, MF479-40) was implemented to generate anoutput spectrum bandwidth (Δλ) of 80 nm, which corresponded to a grismseparation of 108 mm for chromatic correction. Based on the above relaydesign and applying a 100× oil-immersion objective, the separationbetween the three in-focus object planes (Δz) was 839 nm. By configuringthe objective Nomarski prism and/or a quarter-wave plate (whennecessary), the contrast of multi-plane images can be adjusted. Incomparison with phase contrast microscopy imaging mode, DIC will providefiner details of the edge structures in three-plane images withoutartificial halos.

SUMMARY

The combination of 4D multi-plane broadband imaging system with variousmicroscopy imaging modes has formed a solid proof of the effectivenessand the practical applicability of our imaging technique. Due to thesimple, easy-to-use and compact optical set-up, our imaging system caneither be built as an optical attachment which is fully compatible witha commercial microscope and standard camera system, or be integratedinto the optical path of a microscope, and hence customizing a novelmicroscope.

Example 3: 4D Nine-Plane Broadband Imaging

The NRQD grating we discussed above is made up of a series of concentricarc grooves (less than hemi-circular) with varying radii, by which thedetour phase of incident light can be produced, allowing images of threeseparate object planes to be simultaneously obtained and split across asingle image plane. To record images of more planes (up to nine) in asingle snapshot, the mask pattern of an NRQD grating can be designed asan orthogonal combination of two mask patterns of “arc” structure (seeFIG. 3 for the “arc” structure), which is the so-called “crossed”structure (as illustrated in FIG. 19). When the light flux loss does nothave a critical effect on the imaging performance of the optical system,a “crossed” NRQD grating can be replaced by a superposition of two “arc”NRQD gratings that are placed at orthogonal orientation and have thesame design as the corresponding “arc” structures of the “crossed” NRQDgrating. However, the optical efficiency is one of the most importantfactors that affects the imaging performance of our optical system. Tomake effective use of the broadband spectral energy and thus achievesimultaneous nine-plane broadband imaging in high efficiency, we mayhave to focus on the combination mask design of “crossed” NRQD gratingand a simple chromatic correction scheme.

By optimizing the grooves structure of a “crossed” NRQD grating (i.e.curvature, period and etch depth) and centre positions of the two setsof concentric arcs, energy-balanced multi-plane (up to nine) images canbe simultaneously located on the image plane in the form of a 3×3“sudoku” box. Especially when the mask pattern of a “crossed” NRQDgrating is composed of two orthogonally overlaid mask patterns of anidentical “arc” NRQD grating, only five in-focus object planes can besimultaneously recorded; the corner images disappear due to thecancellation of the equal and opposite-sign focal lengths of the twoidentical “arc” NRQD gratings. To simultaneously record nine equidistantin-focus planes, one “arc” NRQD grating should have three times theobject plane separation in the optical system (Δz, see equation (2)) ofthe other “arc” NRQD grating, hence the two “arc” NRQD gratings have thecurvatures (i.e. W₂₀) in the ratio of 1:3 (as illustrated in FIG. 19).For a magnification of 1:1 telecentric 4D multi-plane broadband imagingsystem, the field of view (FOV) at both the image and object planes isonly dependent on the detector's physical size (without aperture) or theaperture itself. Under similar imaging conditions (such as alight-source, microscope set-up, and the imaging quality of a specimen),the combination of our 4D nine-plane broadband imaging system andmicroscopy technique can have a much larger FOV than that of a differentimplementation of similar principles investigated elsewhere (so-calledmultifocus microscopy imaging)—ca. 35×35 microns (60× magnification) orca. 20×20 microns (100× magnification) (S. Abrahamsson, et al. ‘Fastmulticolor 3D imaging using aberration-corrected multifocus microscopy’,Nature Methods 10, 60-63 (2013)).

For a chromatic correction scheme of a “crossed” NRQD grating basedimaging system, two pairs of grisms may be utilized, in which one pairof grisms can be treated in the aforementioned manner, while anotherpair should be rotated on the optical axis by 90° to correct thechromatic dispersion induced by the 90°-orientated “arc” NRQD grating.Based on the ray-path model of the grism system investigated above, thetwo pairs of grisms can be designed using the tabulated functions (suchas FIG. 10, FIG. 17 and FIG. 18), depending on which type of glass ischosen as the grism substrate. In some cases when optical glass with ahigh refractive index is needed, i.e. N-SF11 (SCHOTT) glass, grismparameters can be selected from a tabulated function illustrated in FIG.20. Each grism in the optical system can have an identical design ordifferent parameters, which is highly dependent on the optical relaydesign. And the grisms can have any location in the optical system aslong as the theoretical pre-dispersion and re-collimation for the fullincident waveband can be fulfilled. Here we note that the workingwaveband of grisms and thus the 4D multi-plane broadband imaging systemis not limited in the visible spectrum but can be extended to the domainof invisible light, and the principles of optical design are exactly thesame as we have demonstrated above.

In this invention, the efficiency of the grating is defined as theenergy flow of light diffracted into the orders being measured, relativeto the energy flow of the incident light. Both NRQD gratings and grismswith high efficiency are desirable for 4D multi-plane broadband imaging,especially in measuring/tracking rapidly moving objectives with eitherweak signal or noisy background. For a 4D nine-plane broadband imagingsystem, the limited light flux must be evenly split into nine images,and two pairs of grisms will be applied for correcting the chromaticdispersion of broadband light induced by the NRQD grating. Therefore,the imaging performance of this optical system is highly dependent onthe improvement of the efficiency of each optical element and thereduction of energy loss of the optical system. Grating efficiencyimproved by multi-etch fabrication, which achieves a multi-level phaseprofile of a grating, has been discussed before (Y. Feng, ‘Optimizationof phase gratings with applications to 3D microscopy imaging’, adoctorate dissertation, University of Science and Technology of China,2013). To obtain higher optical efficiency, an NRQD grating can befinely fabricated to achieve a multi-level (digitised) orcontinuous-level (analogue) profile structure. Further, the opticalefficiency of a dispersion compensation set-up can be optimized by avolume phase holographic (VPH) grism instead of a normal grism (agrating combined with a prism), in which case a holographic grating issandwiched between two prisms (Y. Feng, et al. ‘Optical system’, UKPatent Application No. GB2504188-A, (2013)). A prism on each side of theVPH grating provides the correct angles of incidence and diffraction atthe grating and hence maximizes the efficiency. The refractive indicesof the grism components, blazed/VPH grating and prism(s), can bedifferent. In this invention we have found that, in principle, the VPHgrism system can significantly improve optical efficiency, and itschromatic correction performance is very similar to that of the normalgrism system we used in previous applications. By the use of ultra-fastlaser inscription, the grooves of VPH grating may be fabricated at anappropriate angle directly on the prism surface, which would avoid theneed of a second prism and thus reduce energy loss. Furtherinvestigation of the chromatic correction scheme is still in progress.

1. An apparatus for four-dimensional multi-plane broadband imagingcomprising: one or more non-reentry quadratically distorted (NRQD)gratings, which are defined as QD gratings without re-entrant grooves,arranged in a multi-element optical system to produce a focal length anda spatial position associated with each diffraction order; one or morepairs of grisms arranged to manipulate the optical path in space bywavelength for correcting the chromatic dispersion of a broadband inputbeam generated by the NRQD grating(s); a lens system arranged toeffectively modify the focal length of the optical system associatedwith each diffraction order of the NRQD grating(s) and manipulate theoptical path to meet the design requirements of the grism system and,means for light detection.
 2. An apparatus according to claim 1 whereinthe NRQD grating is designed by a combination mask pattern whichcomprises more than one NRQD arc pattern such that the in-focusmulti-plane (more than 3) images can be spatially arranged on a singleimage plane.
 3. An apparatus according to claim 1 or 2 wherein the NRQDgrating has a multi-level (digitised) or continuous-level (analogue)profile structure.
 4. An apparatus according to claim 1, 2 or 3 whereina variety of NRQD grating types can be utilized, which consist ofalternate regularly spaced grooves of different transmissivity,reflectivity, optical thickness or polarisation sensitivity.
 5. Anapparatus according to claim 1 wherein the grisms are volume phaseholographic (VPH) grisms.
 6. An apparatus according to claim 1 whereinthe design of a grism can be defined by its groove density, which can bespecified from 100 to 800 lines per mm when the refractive index of thegrism substrate is 1.4-1.5.
 7. An apparatus according to claim 1 whereinthe design of a grism can be defined by its groove density, which can bespecified from 100 to 900 lines per mm when the refractive index of thegrism substrate is 1.5-1.6.
 8. An apparatus according to claim 1 whereinthe design of a grism can be defined by its groove density, which can bespecified from 100 to 1200 lines per mm when the refractive index of thegrism substrate is 1.6-1.7.
 9. An apparatus according to claim 1 whereinthe design of a grism can be defined by its groove density, which can bespecified from 100 to 1400 lines per mm when the refractive index of thegrism substrate is greater than 1.7.
 10. An apparatus according to claim1 or 5 wherein the refractive indices of the grism components, gratingand prism(s), can be different.
 11. An apparatus according to claim 1, 2or 5 wherein more than one pair of grisms can be utilized such that thechromatic dispersion of a broadband input beam generated by more thanone NRQD grating can be corrected.
 12. An apparatus according to claim1, 2 or 5 is compatible with multi-mode commercial microscopes includingfluorescence, bright/dark field, phase contrast, differentialinterference contrast (DIC) and structured illumination.
 13. Anapparatus according to claim 1, 2 or 5 is versatile enough to combinewith various modern techniques including microscopy, astronomicaloptics, optical data storage, biomedical imaging, wavefront analysis andvirtual/augmented reality.